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[57] ABSTRACT 

A remote control device for a home entertainment system 
has a GUI with touch screen functionality. The visible part 
of the GUI is hierarchically organized in several control 
levels. All levels have consistent lay-outs with a first area for 
selectable items in a first location, a second area with 
controls for the selected one of the items in a second location 
and a third area for content data relating to the selected item 
in a third location. The spatial relationship between locations 
and areas is maintained throughout the levels. Such a GUI 
contributes significantly to the user-friendliness of the home 
entertainment system. 

16 Claims, 6 Drawing Sheets 
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220: BAND WITH GRAPHICAL REPRESENTATIONS OF BASIC INFORMATION PANELS; 
222: BAND WITH GRAPHICAL REPRESENTATIONS OF BASIC CONTROLS; 

224: RACK WITH LINEAR ORGANIZATION OF GRAPHICAL REPRESENTATIONS OF SELECTABLE ITEMS IN LAYER 202;' 
802: ARROW ICON FOR ACTIVATING PANEL 252 IN FIG. 2; 
806: RETURN SOFT-KEY FOR RETURNING TO HOME LEVEL. 
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202, 204, 206: LAYERS IN HIERARCHY; 

208, 210, 212, 214, 216, 218: GRAPHICAL REPRESENTATIONS OF SELECTABLE ITEMS IN LAYER 202; 

228, 230, 232, 234, 236, 238, 240: GRAPHICAL REPRESENTATIONS OF SELECTABLE ITEMS IN LAYER 204; 

220, 242, 248: BANDS WITH GRAPHICAL REPRESENTATIONS OF BASIC INFORMATION PANELS; 

222, 244, 250: BANDS WITH GRAPHICAL REPRESENTATIONS OF BASIC CONTROLS; 

224, 226, 246: RACKS WITH LINEAR ORGANIZATION OF GRAPHICAL REPRESENTATIONS OF SELECTABLE 

ITEMS IN LAYER 202, 204 AND 206, RESPECTIVELY; 

252, 256, 258: SUPPLEMENTARY PANELS FOR SUPPLEMENTARY CONTROLS; 

254, 260, 262: SUPPLEMENTARY PANELS FOR SUPPLEMENTARY INFORMATION; r i r\ 
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246: RACK WITH LINEAR ORGANIZATION OF GRAPHICAL REPRESENTATIONS OF SELECTABLE ITEMS IN LAYER 206- 
248: BAND WITH GRAPHICAL REPRESENTATIONS OF BASIC INFORMATION PANELS- 
250: BAND WITH GRAPHICAL REPRESENTATIONS OF BASIC CONTROLS- 
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220: BAND WITH GRAPHICAL REPRESENTATIONS OF BASIC INFORMATION PANELS 
222: BAND WITH GRAPHICAL REPRESENTATIONS OF BASIC CONTROLS- 
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iu liiN l E/KlAiNMkNT SYSTEM trolling a selected one of the functionalities. The first and 

FIELD OF THE INVENTION second areas stay in their respective locations on the display 

vnmiuN throughout the multiple levels. Preferably, each respective 

The invention relates to a consumer electronics system 5 ° ne ° f the la y~ outs also provides, in a third area of the 

with a plurality of user-selectable and user-controllable ^P^y, a third graphical representation of content data 

functionalities. The functionalities are distributed over mul- relevant to the selected functionality. The third area keeps its 

tiple control levels. The system comprises a display for locati °n on the display throughout the multiple levels as 

displaying a graphical user interface (GUI), and a controller WeU ' thus cons erving to the uniformity of the lay-outs 

coupled to the display for enabling a user to control the 10 throu S hoilt the levels for selectability, controllability and 

system through the GUI. content data. 

BACKPROTTMn adt p lay "° m iD the invention is A™ made spatially uni- 

BACKGROUND ART formly consistent throughout the levels of the control hier- 

Icons are well known in the art of graphical user- 15 cr S onoimc design of the GUI lets the user easily 

interfaces for control of an information processing system specific selectable and controllable items owing to the 

An icon is a small pictorial representation of some larger set fj? clu ? term g °* graphical representations according to 
of information. An icon provides graphical information in a ?! tyP ° ^ functionalit y : selection, control or content 
condensed format about contents or status of the underlying 

M BRIEF DESCRIPnON OF THE DRAWINGS 
or operation of the system in a quick manner. The system is The invention is explained in further detail bv wav of 
accessed or operated upon through actuation of the icon. example and with reference to the accompanying drawing 

An example of a controller unit for a home entertainment wherein: 
system is the Stage 3 Controller unit of Kenwood, described 25 FIG. 1 is a block diagram of a system according to the 
m Kenwood's publicly available manual "STAGE 3/Setting invention; according to the 

the GUI. The GUI provides a large number of icons that FT r T c a n m * c.u 

correspond to a large number of system functionalities. The 30 , u show " 

functionalities are activated through the icons on the touch ^ g ^ ran S ements of graphics user information at various 

screen. The GUI is user-programmable to select the icom! ^ ° f f° ° f *» 

that should be present in the main menu and those that . ^^g 130111 ^e figures, same reference numerals indicate 

should not. In addition, the control modes (e.g., IR control sunilar or corresponding features. 

codes, balancing audio speaker system) of the components 35 nrTATT m 

thus selected can be programmed. DETAILED EMBODIMENTS 

Icons in general are meant to convey information to the First ' ^ s y stem according to the invention is described 

user m a quick manner through an appropriate graphical below * Then » ^ three-dimensional organization of the GUI 

representation. However, if a large number of icons are 15 ex P laine d. 

presented at any of a variety of levels in a hierarchy of icons, 40 FIG 1 » a block diagram of a system 100 according to the 

the user has to actually read, i.e., decipher, the screen's invention. In this example, system 100 is a home entertain- 

mtormation content to locate the desired functionality. As a ment system that comprises multiple top-level sub-svstems 

result, tie user-friendliness of the system control aspects can 102 > *H 106, 108, . . . , 112 with functionalities that relate 

easiiy be degraded if too much information is presented in to the processing audio and video data Sub-system 102 

too many different icon-arrangements. A drawback of the 45 ^eludes a first VCR. Sub-system 104 includes a second 

pnor art control unit discussed above is the inconsistency VCR - Sub-system 106 includes a television set with monitor 

regarding the locations of the icons that require the user to with a cable or satellite TV-tuner. Sub-system 108 

actually study and decipher the menu items displayed, in deludes a laser disk player. Sub-system 110 includes a 

order to find the desired functionality. CD-changer. Sub-system 112 comprises a surroSsomid 

OBJECT OF THFTMVPmttam 50 SPS , louds P eaker ^angement. Each of sub-systems 

OBJECT OF THE INVENTION 102-112 is composed of two or more lower-level sub- 

It is an object of the invention to provide a more user- s >f ems ^at are discussed further below. In this example, 
friendly GUI for control of a home entertainment system sub-systems 102-112 are coupled to one another in order to 

have them cooperate with one another in a controllable 
SUMMARY OF THE INVENTION 55 fasmon - For example, the audio information received 

sr^^isSssS ■ EH - 
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Electronics, can be used as well in combination with appro- 
priate routing control devices. The present invention is 
concerned with the GUI aspects. 

System 100 also comprises a controller 114 and a display 
116. Display 116 provides a GUI. Controller 114 is coupled 
to display 116 for letting the user operate the selectable 
functionalities and controllable functionalities of system 100 
through the GUI. Controller 114 and display 116 are func- 
tionally and physically integrated in a hand-held remote 
control device 118. Display 114 has a touch-sensitive 
functionality, referred to as "touch screen", in order to let the 
user control system 100 by touching the screen at appropri- 
ate screen locations. Other top-level sub-systems (not 
shown) may include, a cassette deck, a radio tuner, a climate 
control system, an artificial lighting control system, a 
camera-based security system, etc. Control device 118 has a 
hard key 120 in order to facilitate sound volume control of 
one or more of sub-systems 102-112 in case the sound has 
to be turned low immediately, e.g., in an emergency when 
the initial setting upon activating one of the devices 102-112 
has an audio functionality and produces a sound that is much 
too loud upon switching it on. Control device 118 may have 
additional hard keys (not shown) for controlling other sys- 
tem functionalities, e.g., channel-up/down. 

FIG. 2 is a diagram of the lay-out of the graphics 
information provided by the GUI at display 116. The GUI is 
organized as a hierarchy of layers 202, 204, . . . , 206, . . . 
Each of layers 202-206 comprises icons for selectable 
sub-systems at the associated level, icons for the corre- 
sponding basic controls, and content data information for the 
associated level. A next one among layers 202-206 is 
accessible through selecting an icon in the preceding layer. 

Top layer 202 comprises graphical representations 208, 
210, 212, 214, 216 and 218 for selectable items, i.e., 
sub-systems 102-112. Top layer 202 comprises a band 220 
with graphical representations of basic information panels 
(no details here) with content data relating to the available 
ones of items 102-112, and a band 222 with graphical 
representations (no details here) of basic controls for each of 4Q 
items 102-112. 

Graphical representations 208-218 of the selectable items 
are arranged linearly as in a rack 224. Graphical represen- 
tation 212 of the selected item, e.g., television receiver 106, 
is located at the center of rack 224. Graphical representation 45 
212 of the selected item in the center is preferably repre- 
sented in a manner that visually contrasts with the repre- 
sentations of the non-selected items. For example, the 
selected item may be graphically represented by an icon that 
is larger than the icons for the non-selected items in rack 
224, thus suggesting a sense of depth. Alternatively, or 
supplementary, the icon of the selected item is shown against 
a background that is colored or shaded differently from that 
for the non-selected items, or it is made to flash, etc. 

The user selects a particular one among items 208-218 in 
rack 224 by touching display 116 at the corresponding 
location. Rack 224 then spins to put the selected item at the 
center of the visible portion of rack 224. The displayed 
portion of basic contents information band 220 and basic 
controls band 222 then change accordingly. Alternatively, 
the user touches and holds a scroll button at display 116 to 
move rack 224 in a particular direction until the desired 
selectable item is at the center. In the latter case, rack 224 has 
fixed end points. These fixed end points may make it easier 
to remember the location of items 102-112 on rack 224, and 
prevent the user from overshooting the end of the rack. The 
graphical representation of the most commonly used ones of 
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selectable items 208-218 (e.g., the one that represents TV 
tuner 106) is located at the center of the complete rack 224 
by default. Preferably, the user has the option of re- arranging 
this default configuration by programming controller 114, 
e.g., to match the physical location of sub-systems 102-112 
in his/her room. 

Rack 224 with graphical representations 208-218 of the 
selectable items, basic information band 220 and basic 
controls band 222 lie in the same plane. Basic information 
band 220 in this example is arranged between device rack 
224 and the top of the screen of display 116, and basic 
controls band 222 is arranged between device rack 224 and 
the bottom of the screen. Band 220 will thus not be blocked 
from view by the user when he/she is manipulating the basic 
controls in band 222: 

Next lower level 204 is reached through top level 202 by 
activating, in this example: touching, icon 212 that is 
currently located in center of the visible portion of rack 224. 
Next lower level 204 has a similar lay-out as layer 202. That 
is, layer 204 comprises a rack 226 of selectable items 
228-240 flanked by a basic information band 242 and a 
basic controls band 244 in a configuration similar to the one 
of layer 202 as discussed above. Selection of a particular one 
of items 228-240 is done in a similar manner as discussed 
with reference to top layer 202. The displayed portions of 
basic information band 242 and basic controls band 244 
change accordingly. Getting back to the next higher layer is 
achieved through a return icon that is uniform for all lower 
levels 204, . . . , 206. The relationship between layer 202 and 
layer 204 is explained in more detail below, in the case the 
passage is made through selecting icon 212 in the center rack 
224. Icon 212 stands for one of top-level sub-system 
102-112. Each of top-level sub-systems 102-112 is com- 
posed of multiple lower-level sub-systems, also referred to 
as functionalities throughout this text. Layer 204 emerges 
when icon 212 in layer 202 is activated. Layer 204 now 
displays selectable items 228-240 that represent lower-level 
sub-systems of the top-level sub-system represented by icon 
212. In this example, the top-level sub-system is TV receiver 
106 and, here, its corresponding sub-systems are the tunings 
to the available TV channels. If icon 212 had represented 
CD-changer 110, sub-systems 228-240 would be the repre- 
sentations of the available CD's stored. 

Next lower levels such as level 206 have a lay-out similar 
to that of level 202 or 204. 

Control level 202 provides supplementary panels 252 and 
254 that are made to appear to slide from the edge of display 
116 over rack 224 and back again upon touching an appro- 
priate key or icon (not shown here). Supplementary panel 
252 provides supplementary controls, in addition to the 
basic controls in the visible portion of band 222, for the 
sub-system currently in the center of rack 224, i.e., sub- 
system 212. For TV sub-system 106 supplementary controls 
comprise, for example, soft keys to adjust contrast, 
brightness, hue, etc., keys to trigger picture-in-picture, 
closed captions, mosaic, etc. Supplementary panel 254 pro- 
vides supplementary data content information, e.g., a text 
summary of the TV program currently being broadcasted on 
the channel that TV system 106 is tuned into. This kind of 
information is made available, e.g., by means of an elec- 
tronic program guide (EPG). Levels 204 and 206 have 
similar supplementary control panels 256 and 258, 
respectively, and similar supplementary content data panels 
260 and 262, respectively. Further details are given below. 

FIG. 3 gives an example of the GUI when TV receiver 
106 has been selected. Graphical representation 212 of TV 
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^ » e i ectronic PWn guide at top level 202 to enableSog tol 2 feveU^ 
(EPG). Also, the visible portion of basic information band play a video game on the display of the remote Retwn key 

«1 u- J"? datC ° f ** CUrrent ^ 806 iS °«"y « the same location t^goonUeS 

cur ent ume throughout the levels. Graphical representations 10 202-206 and has a uniform appearance for instant recoS- 
for the basic controls of TV receiver 106 are being displayed ^n. See FIGS. 3-*. m 

o^nT^cZn^ ° f band ^ hCre ^ f ° r 2^?™^ aPPCaring thr ° Ugh0Ut «^ levels 
CDangmg tne channels. 202-206 is a record soft key 808. See FIGS. 3-8 

WU 4 shows the GUI wherein information panel 220 is Traditionally, users have controlled home entertainment 
aJso used to give temporarily visual feedback to the user, ^5 systems with several remote control devices When they 
e.g when manipulating the basic controls in band 222 or t0 record some content, they would need to find the 

Hard key 120 on remote control device 118. For example, appropriate remote control device, find the record button on 
wnen the user is adjusting the sound volume of TV receiver ll > * nd ** U P the destination device to receive the recording 
106 through hard (physical) key 120 on device 118 or via an nis hold * true for both hard-button-only remote control 
appropriate soft key, rather than putting the visual feedback 20 devices and touch-screen-based remote control devices such 
on the monitor of TV receiver 106 and degrading the as the one discussed in the background art section above 
viewing experience, the visual feedback appears temporarily ^ invention now provides a GUI that simplifies the 
on display 116 of remote control device 118, e.g., as a slider recording process for all sub-systems by integrating the 
SCale 402 recording function into one button, here soft-key 808, which 

FIG. 5 illustrates the case wherein the selected item is 25 * dways availablc ™ the same location regardless 7 of the 
CD-changer 110, its graphical representation 216 being sub-system currently selected. Some of devices 102-112 
shown in the visible portion of rack 224. The visible portion functl0n 35 data source (e.g., laser disk 108), some function 
of basic information band 220 presents information about aS data destmatio A (e.g, loudspeaker system 112 ), some can 
the current CD: for example, the disc number, type, track ?£i?^ oth roles (eg > ^ receiver m > VCR's 102 and 
number, title of the track, and time elapsed. This information 30 m) WheD a sele cted one of items 208-218 or 228-240 
changes automatically when the user switches to another ^ resentS a (P ossible ) data source, touching record button 
CD, or browses within a CD. Visible portion of basic 8 makes destinat i°n sources available for selection as 
controls band 222 now provides the basic controls of recordin g devices by the user. More specifically, a user 
CD-changer 110: stop, play, next/previous disc, next/ browses to the content that he/she wants to record, and then 
previous track, eject, etc. 35 touches record button 808. The GUI then responds by 

FIG. 6 is a diagram of the GUI at next lower level 204 Somatically presenting a rack of sub-systems that he/she 
When the user touches graphical representation 212 of TV ^ V°' the to be recorded. For 

receiver 106 in rack 224 at level 202 as shown in the diaeram fif n u USCI t0Ucbes the record button 808 * ™ G 3, 
of FIG. 3, the GUI animates toward next lower level 204 and u I ^ t0 the ° De ° f FIG ' 9 ' Record buUon 808 *™ 
appears as in FIG. 6. Rack 226 now represents as selectable 40 !?? g t0 f ? ? K " ic0n 902 S y stem 100 has determined 
items a list of TV channels identified by their names, and T acceptoble destinations are VCR's 102 and 104, given 
also by numerals displayed with each name The soft keys in T ^ content > 10 this case a video signal, and given the 
the visible portion of basic controls band 244 are now used devices that m ^ to be present. Presence is determined 
to select one of the channels by their number in order to iiV'iV USCr P rc -P r °g rammin g remote control device 
have displayed in the visible portion of basic'information 45 u \ A ™ nM ^y v *3Wtem 100 keeps an inventory itself of 
band 242 information relating to the program being cur- 7 tt * ub ~ s y stems have been Pegged in. Reference is made 
rently broadcasted at a channel, other than the one that has A ° a PP lication Scr - No - 08/731,624 of same 

been selected and is displayed in the center of rack 226 « lg 5 ^ mentloned above - Alternatively, controller 114 
FIG. 7 is a diagram of next lower level 206 that ™n« '„ P I T ^f™* P resent ^ d then which ones 

when the user touches the. <*nL nf «^ :J^ P ^^ 50 ^ ™ ates for the recording. Mere data sources such as 
FIG. 6„ .eve, 204. R.d, £1™ « S^SSTJl"^ 2 °f "f™/™ 

246 and the day selected in the visible portion of |2 2M sub-system appears m the center of rack 

controls band 250. - ' 



is placed b, pud 252 ofi™lt™„^L?m ™^' h f»»Onc,,b™ 1 ,of St k a ,bki, e ™224.226 
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trols bands 222, 244 and 250 are then preferably located at 
the right hand side of vertical band 224, 226 or 246, if the 
user is right-handed, and at the left side of vertical band 224, 
226 or 246 if the user is left-handed. A suitable program- 
mability feature could be provided for implementing select- 
ability. For example, a home page of the GUI, that can be 
reached through return button 806 at level 202 should 
provide a personalized entrance for each individual user, 
e.g., an icon with the user's name, who then can pre-program 
his/her preferred GUI layout. 
We claim: 

1. For use in a consumer electronics system with a 
plurality of user-selectable and user-controllable function- 
alities distributed over multiple levels in a control hierarchy, 
a hand held remote control device comprising: 

a display for displaying a graphical user interface (GUI); 
a controller coupled to the display for enabling a user to 
control the system through the GUI; wherein: 
the GUI provides a respective lay-out for each respec- 
tive one of the multiple control levels; 
each respective one of the lay-outs provides: 

in a respective first area of the display, first graphical 
representations of functionalities selectable at the 
respective level; and 
the respective first areas occupy a first portion of the 
display at substantially the same first location 
throughout the levels, and wherein: 
each respective one of the lay-outs comprises, in a 
respective second area of the display, second 
graphical representations of basic controls for 
controlling a selected one of the selectable func- 
tionalities at the respective level; and 
the respective second areas occupy a second portion 
of the display at substantially the same second 
location, different from the first location, through- 
out the levels. 

2. The remote control device of claim 1, wherein: 
each respective one of the lay-outs provides, in a respec- 
tive third area of the display, a third graphical repre- 
sentation of content data relevant to the selected func- 
tionality at the respected level; and 

the respective third areas occupy a third portion of the 
display at substantially the same third location, differ- 
ent from the first location, throughout the levels. 

3. The remote control device of claim 1, wherein the 
respective first area comprises a band substantially across 
the display. 

4. The remote control device of claim 1, wherein the 
respective first area comprises a band substantially across 
the display; and the respective second area is located 
between a side of the display and the band. 

5. The remote control device of claim 4, wherein: 
each respective one of the lay-outs comprises, in a respec- 
tive third area of the display, third graphical represen- 
tations of content data relevant to the selected func- 
tionality at the respected level; 

the respective third areas occupy a third portion of the 
display at substantially the same third location, differ- 
ent from the first and second locations, throughout the 
levels; and 

the respective third area is located between another side of 
the display and the band. 

6. The remote control device of claim 1, providing hard 
controls for control of a sound volume produced by an audio 
functionality in the system. 
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7. The remote control device claim 1, wherein each 
respective next one of the lay-outs has a further graphical 
representation of a return button for return to the preceding 
higher level, the further graphical representation being uni- 
form throughout the levels with regard to location and 
appearance. 

8. The remote control device of claim 1, wherein each 
respective lay-out has a further graphical representation of a 
selectable record button that enables recording upon selec- 
tion of a recording functionality in the system and that is 
uniform throughout the levels with regard to location and 
appearance. 

9. A consumer electronics system with a plurality of 
user-selectable and user-controllable functionalities distrib- 
uted over multiple levels in a control hierarchy, the system 
comprising a control device for user-control of the system, 
the system comprising: 

a display for displaying a graphical user interface (GUI); 
a controller coupled to the display for enabling a user to 
control the system through the GUI; wherein: 
the GUI provides a respective lay-out for each respec- 
tive one of the multiple control levels; 
each respective one of the lay-outs provides: 

in a respective first area of the display, first graphical 
representations of functionalities selectable at the 
respective level; and 
the respective first areas occupy a first portion of the 
display at substantially the same first location 
throughout the levels and wherein: 
each respective one of the lay-outs comprises, in a 
respective second area of the display, second 
graphical representations of basic controls for 
controlling a selected one of the selectable func- 
tionalities at the respective level; and 
the respective second areas occupy a second portion 
of the display at substantially the same second 
location, different from the first location, through- 
out the levels. 

10. The system of claim 9, wherein: 

each respective one of the lay-outs provides, in a respec- 
tive third area of the display, a third graphical repre- 
sentation of content data relevant to the selected func- 
tionality at the respected level; and 

the respective third areas occupy a third portion of the 
display at substantially the same third location, differ- 
ent from the first location, throughout the levels. 

11. The system of claim 9, wherein the respective first 
area comprises a band substantially across the display. 

12. The system of claim 9, wherein the respective first 
area comprises a band substantially across the display; and 
the respective second area is located between a side of the 
display and the band. 

13. The system of claim 12, wherein: 

each respective one of the lay-outs comprises, in a respec- 
tive third area of the display, third graphical represen- 
tations of content data relevant to the selected func- 
tionality at the respected level; 

the respective third areas occupy a third portion of the 
display at substantially the same third location, differ- 
ent from the first and second locations, throughout the 
levels; and 

the respective third area is located between another side of 
the display and the band. 
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